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Abstract 

We introduce the title process via a particular construction, and relate it to processes previ- 
ously studied, in particular a process introduced by G. E. H. Renter in 1969. We derive elemen- 

tary properties and quantities of this processes: Markov property, transition rates, stationary 
distribution, and the infinitesimal generator for a case not treated by Renter. 
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4^' 1 Introduction 

^ The aim of this paper is to introduce and analyse a class of Markov processes on a countable state 

space, which we will call K processes. This terminology has been used before in Pontes and Mathieu 
(2008) for a subclass of the class we consider here. In comparison, we might call the processes of 
the present paper K processes with weights, and the processes previously studied in that reference, 
^ K processes with uniform weights. 

Let us start with a tentative description of our process. Let S denote a countably infinite set 
^ — our tentative state space. We want our process to do the following: when at x G S, it waits an 
exponential time of mean 7^,, and then jumps to site y G S with probability proportional to Xy, 
where A := {X^, a; G S} is a set of weights. This would make immediate sense once we assumed 
that the sum of the weights in A — the total weight of S — is finite. 

But we make the opposite assumption, namely that the total weight of S is infinite. This is 
natural in the context of scaling limits of trap models — more about those models below. Since S 
is infinite, that assumption will then force a condition on the mean waiting times {jxj x and 
we further need to introduce an extra point in the state space. 

A more precise, even if still rough description of our process is as follows. Let 00 denote the 
above mentioned extra point. Our process lives on S = S U {00}. When at a:; G S, it waits an 
exponential time of mean jx and then jumps to 00, which is an instantaneous state. Starting at 00, 
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the process enters finite sets F G S with distribution proportional to {A^;, x E F}. We assume our 
parameter set {73;, Xx, x e F} satisfies 7a;, A^; > for each x and 



There is a further parameter c > related to the size of the 00-set of the process (i.e., the set of 
times that the process spends visiting 00). A precise definition/construction of the process will be 
given in the next section. 

The above mentioned connection with trap models is as follows. K processes arise as scaling 
limits of those models under suitable scaling. This was established in Pontes and Mathieu (2008) 
for the (symmetric) trap model on the complete graph, where the uniform weight case Xx = 1 was 
introduced and studied. More recently, the K process with a particular set of weights was introduced 
and shown to be the scaling limit of asymmetric trap models on the complete graph in Bezerra et al. 
(2012). It also appears as a scaling limit of the trap model studied in Jara et al. (2012). All the K 
processes showing up in this context have c = 0. 

The c > also has a history, albeit a different, older one. The uniform weight case corresponds 
to the famous Kl example of Kolmogorov (introduced in Kolmogorov (1951)) of a process in a 
countable state space with an instantaneous state. This example has prompted many studies since 
its appearance, in particular Kendall and Renter (1956), where this example was further analysed, 
and Renter (1969), where the weighted extension was introduced and analysed. These studies are 
analytical ones, defined via Q-matrices. In particular, one common interest is in the derivation of 
the infinitesimal generators. 

What we do in this paper is to introduce the K process via a particular construction in Section 2. 
We establish the Markov property in Section 4, with Section 3 dedicated to an auxiliary truncated 
process. Transition rates and stationary distribution are obtained in Section 5 — these derivations 
are absent in Pontes and Mathieu (2008) and Bezerra et al. (2012)). It follows from the analysis 
performed in Section 5 that ours is a construction of the process analytically introduced in Reuter 
(1969); as far as we know, such a construction was missing in the general weighted case. Finally, in 
Section 6, we compute the infinitesimal generator in the case c = 0; the c > case, a considerably 
different one, was done in Reuter (1969). 

We close this introduction with a word about description via Dirichlet forms. Pontes and Mathieu 
(2008) take this point of view for the definition and analysis of the uniform weight K process (be- 
sides also the constructive approach employed in the present paper). This is also possible in the 
general weighted case, but we choose not to exploit it in the present paper. 

2 Construction 

The state space is a countably infinite set § = § U {00}, in which 00 is a symbol not in §. The 
parameters of the K process are a family of strictly positive real numbers {7^;, A^^ : x G S} as well as 
a constant c > 0. We will call {jx : a: G S} waiting time parameters, and {A^; : a; G S} the weights 
of the process. We will impose the restrictions (1.1) to these parameters. 

RertiEirk 2.1. Note that (1.1) implies that mfx^g^x = 0. 




(1.1) 
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The probability space {QjTjF) in which we will construct the K process needs to admit the 
following independent random variables: 

• {Nx : X G S}: a family of independent Poisson processes on M+, where have rate A^;. We 
will denote the marks of iVa; by < erf < erf < 

• {Tq} U {T^ : n € N,a; € S}: a family of i.i.d. random variables, exponential of rate 1. 
Definition 2.1. Let the Clock Process be given by: 

m •■= r^t) := 7yn + E E ^^^i + (2-1) 

for t >0 and y £ S. We adopt the convention that 700 = and Yl^=i IzT^ = 0. 

Remark 2.2. {T{t) : t > 0} is a.s. strictly increasing and cddldg. It also has stationary and 
independent increments. For each t > 0, T{t) has finite mean and its distribution is absolutely 
continuous with respect to the Lebesgue measure. 

Definition 2.2. The K process with initial state y £ E> is a stochastic process {Xy{t) : t > 0} 
defined as follows: 



X{t) := Xy{t) := < 



y, ift<-fyTo; 

X, z/tGU=i[rn<-),rn<)); (2-2) 

oo, otherwise. 



RemEirk 2.3. This definition also makes sense when the first sum in (1-1) above is finite, but in 
this case the resulting process is a Markovian pure jump process. This follows from the fact that 
in this case the set of Poissonian marks {af : i > l,x E E^} is discrete (i.e., has no limit points) 
almost surely. 

Remark 2.4. Note that, by construction, the process started at a site y £ S stays there till the time 
7yTo and afterwards behaves as a copy of the K process started at oo. That is: 

xy{t + jyTo) = x^{t), 

almost surely for every t >0. 

RemEirk 2.5. It is a straightforward exercise to check that X satisfies the rough description given 
in the introduction (see beginning of paragraph of (1.1)^, and that c can be obtained as times the 
expected length of the oo-set of X between two consecutive visits to x, for any x G S. 

We will equip S with the following metric: 

d{x,y) = {^x + ^y)l{x^y}. (2.3) 

Under this metric {x} is a open set for each x € §, and, roughly speaking, x is close to oo when 
is small. In symbols: x ^ oo if and only if 7^; — > 0. 
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RemEirk 2.6. One readily checks that (S, d) is a Polish space, not necessarily compact. The con- 
tinuous functions in the topology generated by this metric are the set: 



C = {/ : § ^ M : Ve > 03(5 > s. t. \f{x) - /(oo)| < e whenever -fx < 6] . 



(2.4) 



Continuous functions in this topology are uniformly continuous. 

3 Truncated processes 

We will establish the Markov property for the K process in the next section. Our strategy is 
to approximate it by Markov pure jump processes. In this section, we will construct these approxi- 
mating processes in the same probability space as the original K process, and then prove an almost 
sure convergence in Theorem 3.5 below. 

Definition 3.1. For each 5 > 0, let 



Proposition 3.2. The K process is almost surely cadlag. 

Proof. Using Remark 2.4, we need only show that the K process started at oo is cadlag. 

Following Billingsley (1999), fixing a realization of the process, T > and e > 0, let us show 
that there exists Q = Iq < ti < . . . < t^ = T such that w\ti-i,ti) < e for every i = 1, . . . ,N , where 



Let us fix (5 < e/2, so the diameter of S \ S5 is at most e. And take S*! < . . . < Sm an ordering 
of the set {af : x G S5, i > 1, r(a"f ) < T}. This ordering is possible by Remark 3.1, which implies 
that the latter set is almost surely finite for each finite T. 

Finally let us take N = 2M + I, to = tn = T and for z = 1,...,M: %_i = r(S'j-), 
t2i = r(S'i). If t G \t2i-i,t2i), then X{t) is constant in this interval, while if f G [t2i-2, i2i-i), then 
X{t) G S \ S^, so the variation in this interval is at most e. The same occurs in the intervals [to, t\) 
and [tjv-i,tAr)- The argument is complete. □ 

Remark 3.3. By the argument in the proof of Proposition 3.2, the only candidates for discontinuity 
points of the K process are \T{a^),T{a^ —) : i G N,x G S}. Since this a countable set and each such 
point is an absolutely continuous random variable, then every fixed deterministic t > is almost 
surely a continuity point of the K process. 

Definition 3.2. The truncated process at level 5 > with initial state y is defined as follows: 



85 := {x G S : 7^ > 5}, S5 := §5 U {00}. 



Remark 3.1. Note that the second condition in (1.1) implies that ^ 



Xx < 00 for every S > 0. 



w{A) = snp,^,^^d{X'^{t),X'^{s)). 



Xsit) ■.= Xy{t): 



< 



y ift< lyTo 

X */teU=i[rK<-),rK<)) 



(3.1) 



00 otherwise; 



\ 



N.{t) 




(3.2) 



z&s i=l 
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RemEirk 3.4. Since ^^^g^ < oo (Remark 3.1) then the truncated process is a Markov pure jump 
process described as follows: 

• In the case c > 0, from a state x / oo, the process stays an exponential time of mean ^x, after 
which it jumps to oo. There it stays a exponential time of rate ^ '^Zx&s o-nd chooses a new 
state y eSs to jump to with probability proportional to Xy. 

• The case c = is similar, except that in this case the process never visits oo. After exiting a 
state X & Ss it chooses a new state y G S5 with probability proportional to Xy. Note that in 
this case the truncated process never visits 00; it is even true that X°^(0) / 00 a.s. . 

So we can conclude that Xs is Markovian and cddldg. 

Theorem 3.5. converges to X°°{») a.s. in the Skorohod Ji topology as (5 — >■ 0. 

Proof. Following Proposition 5.3 from Chapter 3 of Ethier and Kurtz (1986), wc will prove that, 
for each T > 0, exists increasing bijective and Lipschitz continuous functions Xs : [0,00) — )• [0,oo), 
S > 0, such that almost surely: 

lim sup d {X^ (t), {Xs{t))) = lmi sup \Xs{t)-t\=0. 

These functions may depend on the realization of the process. As the process always starts at 
00, we will stop carrying this index in our notation. 

Fix a realization of the process and a T > and take = < Sg < ... an ordering of 
{0}U{cjf ■.x£8s,i> 1}. This is a.s. possible since X^xeS^ '^^ 

For < e < 5 let us define: 

LI :=min{z> 1 :r,(5i) >r}. 

The set {af : a; G S \ S^, z > 1} is dense, so for e sufficiently small we have that Tf{Sl'^^—) > 
Tf,{Sl) for every i < L|. 

For such e, we define A| : [0, 00) [0, 00) the following way: 

'nsD + j^^gpljlg!) [t - r.is})] if USD < t < USI+'-) 
AKO = I r(5f 1-) - r,(5f 1-) + 1 if r,(5f 1-) < t < r,(5f ^) 

r{si'^)-Usi'^) + t ^t>uslj 

Note that for z = 0, . . .L^: 

xmisi-)) = r{si-), xmisi)) = T{Si), 

while is a linear interpolation between these points. 

Since T{t) > T^{t) for every e > 0, we have that A^(t) > t. Using the linearity by parts and the 
fact that r(c7f ) = r(af-) + j^T,^ and r,(5^*) > T, we find that: 

sup m) -t\< max^{r(Si) - ^51)}. 

0<t<T 0<i<Ll 
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This quantity vanishes almost surely if 6 is fixed and e ^ 0. So, for every (5 > there exists 
e{S) > such that: 

sup \\sit) -t\ <S 

0<t<T 

for every e < e{S). 

We can take e{S) so that it is decreasing in S. So we can take 6{e) as its inverse. Since e{S) > Gid, 
then lime-^o S{e) = 0. Thus: 

sup \Xl {t)-t\<S{e)^0. 

0<t<T 

By construction we have that t G [r,(5i-), r,(5i)) if and only if A^(t) G [r(5i-), r(5^)). So, if 
X £ Ss, we have that X(A^(t)) = x if and only if X^{t) = x. Observing that the diameter of S \ 
is smaller or equal than 26, we conclude that: 

sup diXiXlit)),X,{t))<2S. 

0<t<T 

Making = ^s(^^y we conclude that: 

sup \X,{t)-t\^^0, sup d{X{X,{t)),X,{t)) ^=^0 □ 

o<t<r o<t<T 

Corollary 3.6. For every y G S, T > and almost every realization of the K process, there are 
increasing bijective and Lipschitz continuous functions Xg'^ : [0, oo) — >■ [0, oo) such that: 

X'i\t)>t, 

sup lA^^V) - *l < sup \x'f\t) - t\ ^ 0, (3.3) 

0<t<T 0<t<T 

sup dixyixl^\t)),Xlit))< sup diX'^ixf\t)),X^it))^0. 

0<t<T 0<t<T 

Thus Xg converges a.s. to X^ in the Skorohod Ji topology as (5 — >■ uniformly in y. 

Proof. After fixing a realization and a T > 0, let us take A^ : [0, oo) — > [0, oo) as in the proof of 
Theorem 3.5. 

Take A^°^^ = A^ and for y G § define: 



t, ift< jyTo 

lyTo + \s{t- ^yTo), if t > -fyTo 

Observing Remark 2.4 we conclude that this family of functions satisfies (3.3). □ 



6 



4 Markov property 



Definition 4.1. We will denote the semigroups of the K process and the truncated process by {^t)t>o 
and (^j)t>o. That is, for / : S — )• M we define: 



Proposition 4.1. Let / : S — )• M 6e a bounded continuous function, then ^tf is also a bounded 
continuous function for every t > 0. 

RemEirk 4.2. This result establishes a property which is close to (but is not quite) a Feller property 
of our semigroup. For a semigroup to be Feller, it would have to take continuous functions that 
vanish at infinity into continuous functions that vanish at infinity. This may not be the case for our 
process, depending on the choice of the parameters. 

Proof of Proposition 4-1 ■ Fix a t > 0. If / is bounded, then obviously ^tf is also a bounded 
function. We are left with showing that it is continuous. For that take a sequence of elements 
of S that converges to x G S. We want to show that: 

lim ^tf{xn) = %f{x) (4.1) 

If X 7^ oo, then observing the metric (2.3) we notice that d(xn,x) > ■~fx^{xn = x}, so there must 
exist an no such that Xn = x for every n > no- (4.1) follows immediately in this case. 
If X = oo, we can write: 

mixn) - %f{x)\ = |E [f{x^-{t)) - f{x^m\ < E \fix'"{t)) - fix^im . 

The quantity inside the expected value is bounded (since / is bounded), so if we can show that 
this quantity converges almost surely to zero then we can conclude with the dominated convergence 
theorem. 

Note that d{xn, oo) = 'jxn "^^°°> 0, so for a fixed realization of the process we can take n large 
enough so 7a;„7o < t. In this case, using Remark 2.4, we can write: 

f{X^-{t)) - fiX^it)) = f{X^{t - 7.„To)) - fiX^m 

This quantity vanishes as n — t- oo since t is a.s. a continuity point of the K process (by Remark 
3-3), 'Jxn — )• as n — >■ oo and / is continuous. □ 

Remcirk 4.3. One way to turn the K process into a Feller process if it does not have that property 
in the above formulation, as pointed out to us by Milton Jara, is to add new sites to S other than 
only oo as follows. For concreteness, let us suppose in this remark that S = {1, 2, . . .}. Let Q be the 
set of limit points of {73; : x G S}. Note that & Q by Remark 2.1. The new state space would be: 

S := {(x,7^) : X G S} U {(00,^) : ^ G g}, 
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equipped with the metric: 



1 1 

X y 



+ bx-ly\, (4.2) 



with the convention that ^ = 0. This can be seen as a completion of the metric (4.2) applied to the 

original space. 

For a point {y,g) G §, we (re)define our process as follows: 

r^'^(i) :=gTo + Y,Y.j,T,^ + ct; 

zes i=i 

iy,9), ift<gTo; 



(x,!.), ift G U=i[r^'n<-),r^'n<)); 

(oo,0), otherwise. 



The point (oo, 0) replaces oo in the original formulation, and the new points (oo, g) £ 8, g > 0, 
are states that are never visited by the K process, except when the process starts in one of them; in 
this case, the process never visits any of them again after leaving that initial one of them for the 
first time. 

All results from this paper, suitably reformulated when necessary, hold also for this version of 
the K process. Additionally, under this reformulation, the process satisfies the Feller property. 

A mild complication of this approach is that S could be uncountable, making the characterization 
of the K process via a Q matrix, as will be done in Section 5, cumbersome. That and the heavier 
notation of the new formulation led us to stick to the original definition of our process. 

Lemma 4.4. For t > 0, let Vt := U5g[o^t]{X(s)} be the set of the visited states up to time t. Then: 

#{x G 14 : 7^ > e} < oo (4.3) 
almost surely for every e > and t > 0. 

Proof. Since the events considered are monotonic in e and t, then showing that (4.3) occurs with 
probabihty 1 for any fixed e > and t > is enough to prove the lemma. 
For fixed e > and t > 0, consider the complementary event, that is: 

{#{x G : 7x > e} = oo} (4.4) 

In this event, for any A/" G N fixed, there are states xi,. . . xn such that: 

N N 



eJ^T^i <J2lxiT^' <t. 



i=l i=l 



So we can conclude that, if is a sum of N independent exponential random variables of rate 
1, then the probability of (4.4) can be bounded by the probability of eS]\f being smaller than t. The 
lemma follows from the observation that Sn — >■ oo as A?^ — > oo with probability 1. □ 
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Lemma 4.5. Take f : S 



a bounded continuous function, t > and a set ^ C S satisfying: 



#{x e A : > e} < oo, 



(4.5) 



for every e > 0. Under these conditions: 



sup|*?/(y)-*t/(y)|^0. 



(4.6) 



Proof. Let us first show the convergence without uniformity, that is, we will show that *t/(y) 
*t/(y) as (5 — >■ for any y E A. Let us write 



^if{y) - ^tf{y) = E [f{xy{t)) - f{xy{t))] . 



(4.7) 



Remark 3.3 guarantees that t is almost surely a continuity point of X^. Since convergence in the 
Skorohod topology implies convergence at the continuity points of the limit trajectory and / is a 
continuous function, then f{Xg{t)) f{Xy{t)) a.s. as 5 ^ 0. 

Since / is bounded, the dominated convergence theorem implies that (4.7) vanishes as well as 

This is readily extended to the case where A is finite. Let us suppose from now on that A is 
infinite. 

Take A^^"* as in Corollary 3.6 for some T > t. We can write: 



sup \^lf{y) - ^tf{y)\ = sup |E [/(X|(i)) - f{Xy{t))] I 



< sup 

+ sup 



f{xy{t))-f{xy{x'i'\t))) 



f{xy{xfm-f{xy{t)) 



(4.8) 
(4.9) 



Let us now show that both terms of this sum vanish as 5 — >■ 0. 
Observing (3.3), then for every y a.s. : 



d{xy{t),xy{x^y\t))< sup d{xr{s),x^{x';-'{s)) ^o. 

se[o,T] 



(oo). 



(4.10) 



Since / is a continuous functions, it is also uniformly continuous (sec Remark 2.6). That is, for 
every e > 0, exists a r]c > such that |/(x) — f{y)\ < e whenever d(x,y) < rj^. 
If we fix an arbitrary e > 0, we can bound (4.8) by: 



sup 

yeA 



< sup 



+ 



f{xy{t))-f{xyix'f\t))) 

[f{xy{t)) - f{xy{xf\t)))) i{d{xy{t),xy{x'f\t)) < ve} 

2\\f\\snpF(d{Xy{t),Xy{X^y\t))>rje) 
yeA ^ ' 



<e + 2 



sup d{Xr{s),X^{xf\s))>r], 
,s6[o,r] 
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Observing (4.10), we conclude that this last quantity converges to e as (5 — >■ 0. Since e is an arbitrary 
positive constant we conclude that (4.8) vanishes as (5 ^ 0. 

To compute (4.9), note that each term of that quantity vanishes as (5 — )■ since t is a.s. a 
continuity point of and X^f\t) — )■ t a.s. . Using (4.5) we can take r]{6) a monotonic function 
that vanishes as 5 — > and if we define As := {x & A : < then: 



max 

yeA\As 



E 



f{xyixf\t)))-fixy{t)) 



> 0. 



Since r){6) 0, we may choose S so small that \/r]{S) < t < \g^\t). The latter inequality follows 
from (3.3). 

Using Remark 2.4 and considering in the cases jyTo < \/r}{5) and 7j,To > ^/ri{S), we can write: 



sup 

yeAs 



E 



f{xy{x'i'\t)))-f{xy{t)) 

/(x-(Af (t) - 7,ro)) - f{x^{t - 7,ro)) i{7,ro < vW)}| 



< sup E 
yeAs 

+2II/II sup P(7^ro > vW))- 

y&As 



(4.11) 



Note that sup^^g^^, -fy < r]{5), so sup^g^^ ^{lyTo > \/r]{6)) < e ^ as (5 ^ 0. 

We can bound the first term in (4.11) by: 



E 



sup sup 



f{X^ixl^\t)-s))-fiX^it-s)) 



The random variable inside the expected value is bounded by 2||/||. So if wc show that it goes 
almost surely to zero, then the dominated convergence theorem will tell us that its expected value 
also vanishes. 

Fixing an e > 0, take e' > such that |/(a;) — f{y)\ < e whenever d{x, y) < e'. This is possible 
by the uniform continuity of /. 

As t is a.s. a continuity point of there exists an e" such that d(X°°(t), X°°(t + s)) < ^ 
whenever \s\ < e". 

If we take Sq such that 5 < Sq implies that \/'q{5) < ^ and supo<s<T |'^5°°'*(s) — s| < ^, then 
for every y G S and s < ^Jr]{5)■. 



\X^5\t)- s-t\<\xf{t)-t\ + \s\< sup \\^^\r)-r\ + \s\<e", 

r-e[0,T] 

|t-s-t| = |s| < e". 



So it is valid that: 

.iy)i 



d{X°°{Xf>{t) - s),X~(i - s)) < d{X°^{xf\t) - s),X~(t)) + d(X°°(t),X°°(t - s)) < e'. 
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Therewith we may conclude that if < (5 < 5o then: 



sup sup 



f(X^{xf{t)-s))-f{X^{t-s)) 



< €. 



Since e > was arbitrarily chosen, we may conclude that this quantity vanishes a.s. as 6 0. □ 
Theorem 4.6. The K process is Markovian. 

Proof. Let us fix m > 1, ti < t-2 < ... < tm+i and /i, . . . , fm+i : S — >■ K bounded continuous 
functions. 

As Xs is Markovian: 



= E 



E ihiXsih)) . . . fm{Xs{tm))fm+l{Xs{tm+l))] 
hiXsih)) . . . fm{Xs{tm))^l^^,^tJm+l{Xs{tm)) 



(4.12) 



Remark 3.3 guarantee that ti, . . . ,tm+i are a.s. continuity points of the K process. As conver- 
gence in the Skorohod topology guarantees pointwise convergence at continuity points and each /j 
is continuous, we may conclude that fi{X^{ti)) fi{X{ti)) a.s. as 5 — )• for every i = 1, . . . , m-|- 1. 

Using the dominated convergence theorem we conclude that the left hand side of (4.12) converges 
as (5 —> to: 

E [h{X{tl)) . . . fm{X{tm))fm+l{X{tm+l))] ■ 



Let us write the right hand side of (4.12) as: 



E [hiXsih)) . . . fmiXsitmmtr„+,-Um+liXs{tm))] + eg. 



(4.13) 



Using arguments analogous to those employed in the computation of the limit of the left hand 
side of (4.12), together with Proposition 4.1, we may show that the left hand side of (4.13) converges 
as (5 ^ to: 

E [hiX{h)) . . . fmiXitm))^tr^+,-Um+l{Xitm))] ■ 

We are left with showing that le^l — >■ as (5 — >■ 0. For simplicity of notation, let us denote 
t = tjn and g = fm+i- Using the fact that fi,...,fm are bounded, we may write: 



le^l < const 



E 



^taiXsit)) - ^sg{Xsit)) 



As the random variable inside the expected value is bounded, if we show that it converges almost 
surely to zero, then the result will follow from the dominated convergence theorem. 

If Vt is as in Lemma 4.4, note that there exists t' > such that Xs{t) G Vf for every 6. Thus 



^ig{Xs{t))-^sgiXs{t)) 



< sup 



Lemma 4.4 guarantees that the condition of Lemma 4.5 is valid for almost every Vj/, so the 
above quantity vanishes a.s. as 5 — )• 0. 
Thus we can conclude that 



E [fliXih)) . . . fmiXitm))fm+liXitm+l))] = E [h{X{h)) . . . /^(X(i^))*t^^,_t^/^+l(X(t^))] , 
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and the proof is complete. □ 

5 Transition probabilities 

Definition 5.1. For x,y gE> and t >0, let us define: 

p,y{t):= Fix- it) = y). (5.1) 

Proposition 5.1. For every x, y G S, Pxy{*) is a continuous function on (0, oo). 

Proof. This is one of the items of Theorem 1 from Section II of Chung (1967). □ 

Remark 5.2. Note that Proposition 5.1 states that the transition functions are continuous outside 
of the origin. It is easy to prove that Pxx{t) 1 as t\0 for every x G S; a harder problem arises 
when X = oo. 



Proposition 5.3. If c> 0, then: 



limpoooo(i) = 1- (5.2) 



Proof. Consider the function: 

0{t) ■= [ I{X°°(s) = oojds. 
Jo 

Since r°°(s) ^ as s \ (Remark 2.2), then for a fixed t > there exists an s > such 
that P(r°°(s) < t) > 0. By construction, on this event, 9{t) > cs. Prom this we can conclude that 
E[0(t)] > 0. Applying Fubini's theorem we obtain that: 



< E[e{t)] = [ Poooo{s)ds 
Jo 



Therefore the set {t > : Poooo{t) > 0} has a positive Lebesgue measure, so it is not empty. Let 
us fix an s > such that Poooo(s) > 0. 

Take an arbitrary sequence (t„), such that tn \ and Pooooitn) converges to a real number u 
as n ^ oo. If we show that u = 1, then (5.2) follows. 

Using the Markov property and Proposition 5.1, we can write: 

^•0000(5) = lim Poooo(s + in) = lim y^Poox{s)Pxoo{tn) ■ 

n->oo n->oo ■'^ 

xes 

By Remark 5.2, if x G S then Pxoo{in) — >■ as n — >■ 00. Since each term in the sum can 
be bounded by Poox{s), which is summable, we can apply the dominated convergence theorem to 
conclude that: 

J'oooo(^) ~ Poooo{s)u. 

Since Poooo(s) > Oj we conclude that u = 1. □ 
Proposition 5.4. If c = 0, a; G § and t > 0, then: 

Pxoo{t) = 0. 
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Proof. For (5 > we define: 

Osit) := j\{X^{s)^^5}ds 
Jo 

Note that Os{t) is monotonic in S. Thus, if S is the right-continuous inverse for T°° then: 

m 

eo{t)<es{t)=Y,Y.^uTr. 

As 5 \ we are summing over the tail of a convergent series, thus Um5\^o^(5(i) = a.s. , so 
= a.s. and E[^o(0] = 0- Using Fubini's theorem we can write 



= E 



/ I{Xo°°(s) = oo}c?sl = / P{Xo~(s) = oo}ds= / Poooo{s)ds, 
Jo J Jo Jo 



from which we conclude that Poooo(i) = for Lcbesgue almost all point t > 0. Using the continuity 
of 

Poooo (Proposition 5.1) and Remark 2.4 we conclude this proposition. □ 

Transition rates Let us consider the transition rates of the K process, given for x,y by: 

_,,_ Pxy{t)-l{x = y} 

^"^"1™ t 

Most of the remainder of this section will be dedicated to proving that these limits exist and 
computing their values for different choices of x and y. At the very end we will use that information 
to give the invariant measure for the K process. 

Proposition 5.5. For every x G S and A cS \ {x} such that YlyeA \ < oo-' 

^^F(X-'it) e A) ^ (5.3) 

As a particular case, taking A = {y}, y G S \ {x}, we have that Qxy = 0. 

Proof. Let us denote by a a = i^^yeA this is an exponential random variable of rate ^y^^. \- 
Using Remark 2.4, we can write: 

t t Jo 7a; 

1 /•* 

< — / P(r~(CTA-) < t - s)ds 

tlx Jo 



= [' F{r^ {aA-)<s)ds. 

tlx Jo 



Note that F{T°°{aA—) = 0) = 0, so the probability inside the integral vanishes as s \ 0. □ 
Proposition 5.6. For every x G §.• 

qxx = • (5.4) 

Ix 
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Proof. Using Remark 2.4, we can write: 

- 1 _ Fi^^TS >t)-l 1 



If 1 

+ - / F{X°°{t-s)=x)—e~ids 

t Jo Ix 



^ 

e 



11/ / ^ 1 , 

— + 7 / Pooa;(i-s)— e 



The first term of tlie latter sum converges to — as t \, 0. Using arguments analogous to the 
ones used in the proof of Proposition 5.5, we can show that the second term of this sum vanishes 
as i \ 0. □ 

Proposition 5.7. For x G S.- 

ifoO 
if = 0. 

Proof The case c = is trivial using Proposition 5.4. So let us suppose c > 0. 

Pxoo{t) 1 



= - / F{X^{t- s) = oo)—e~ ids 

t Jo Ix 

1 /■* 1 _^ 

T / Poooo{t-s) — e t^ds 

t Jo Ix 

/ Poooo{s)e^ds. 
Jo 



e-tl rt 

Ix t JO 



Proposition 5.3 guarantees that Poooo(t) — )• 1 as i \ 0. So the above quantity converges to 
as t \ 0. □ 

Proposition 5.8. For x G S.- 

^ ifoO 

c ■' 

oo if c = 



Proof. 



^ f(r-'W-)<t) _ P(r~W)<t) ^ 1^ /y _) < ^ < r<.(,f (5.5) 
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Working with the second term of this sum, we obtain: 

\ yi^x 1=1 / 

\ yi^x i=\ j 

1 / '^^^'^^^ \ 1 



The probabiHty inside the last integral vanishes as s — >■ 0, from which we can conclude that the 
second term of (5.5) vanishes as t \ 0. 

Note that the event in the third term of (5.5) is contained in the event of the second term, so 
the third term also vanishes as t \ 0. 

Now we arc left with computing the limit of the first term. We are going to do this using 
Karamata's Tauberian Theorem, that relates the behavior of the distribution function of a positive 
random variable near the origin with the behavior of its Laplace transform near infinity. We will 
follow the statement of this theorem given in Feller (1971) (Theorem XIII. 5.1). 

Conditioning in the value of &^ and using the independence of the Poisson processes, we can 
compute the Laplace transform of r°°(c7f — ) by: 



y^x 



Let us first treat the case c > 0. Note that in this case: 

lini '^("^) = i 
a->oo (j){a) P 

So the condition of the Tauberian theorem is verified, from which we obtain that, when c > 0: 

^.^p(roo(.f )<t)^^ 

t\o 0(1) 

Note that ^ ^ as t \ 0, from which we conclude the case c > 0. 

To treat the case c = 0, note that our construction allows for a coupling of several K processes, 
with different values of c, using the same Poisson processes and exponential random variables. So 
let us attach an index c in Fc to denote what is the value of c to which we are referring. 

Note that Fc is increasing in c, so P(F^ (erf—) < t) is monotonic in c. Finally we can conclude 
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that, for every c > 0: 

ii^^tP(rs°K-) < t) ^ i^^nr^K-) < t) ^ A, 

t\o f t\o t c 

Taking small values of c > 0, we conclude that ^ ° " oo. □ 
Proposition 5.9. 

Proof. The case c = is trivial using Proposition 5.4. So let us suppose c > 0. 
Taking {xi,X2, . . .} an enumeration of §, we can write: 

^ iGS ^ 1=1 ^ 

The last term converges to — Yl^=i ^ as t \ by Proposition 5.8, and this goes to — oo as 
n ^ oo because of the first condition in (1.1). □ 

RemEirk 5.10. The above results on the transition rates of the K process, adjoined to the Markov 
property, and the uniqueness result of Reuter (1969), imply that in the c > case, the K process is 
a version of the process introduced in the latter reference. In the notation of that reference, we have 
the following correspondence with the notation of the present paper: 

ax = I/Tx, bx = Xx/c. 
Theorem 5.11. The K process has a unique invariant probability measure, given by: 



Tr(x) 



L c+Ej/gs ^y^y 



Proof. (Sketchy) 

Let us first prove the existence and uniqueness of the invariant measure, using a standard 
argument. For a fixed h > consider the discrete time Markov Chain (y^)„gis} given by: 

YJ::=X{nh). 

Note that this discrete time Markov Chain is irreducible in the case c > and that there is only 
one closed class of communication, namely S, in the case c = 0. Furthermore, in both cases, every 
state X G S is positive recurrent. 

This implies that, for every choice of /i > 0, there exists a unique invariant probability measure 
Hh for (Yj^)n. The Markov property yields that this probability measure is the same for every 
rational choice of h. 

Finally using the continuity of the transition functions (Proposition 5.1) we conclude that fj,h 
is the same for every choice of h. It follows that this is the unique invariant probability measure of 
the K process. 
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Let us thus drop the sub-index h and refer to this probabiHty measure simply as fi. For computing 
it in the case c > 0, note that it needs to satisfy: 



for every t > 0. Taking the Hmit t \ 0, we conclude from the above results about the transition 
rates of the process that for y G S: 

Koo)qooy + Ky)<lyy = 0- 

Using Propositions 5.6 and 5.8, we conclude that the only solution for this system of equations 
that is a probability measure is tt given in (5.7). 

Since Pxyi^) varies continually as a function of c for every fixed x, y G S and t > 0, as can be 
shown via a straightforward coupling argument, we may obtain the case c = by taking the limit 
0. □ 

6 Infinitesimal Generator 

As pointed out in the introduction, the case of positive c was treated in Renter (1969). We will 
restrict our attention in this section to the case where c = 0. 

We will denote by Cq the set of bounded continuous functions. We will show below that the 
following subset of Co is a core for the generator of the K process. 

-j^^L^Co: ^^^^ '•^^''^ " ^ - /(oo))^. = 0, 1 

\ lima;^oo exists J 

Let us recall that in (S, d), x — > oo is equivalent to 'jx — >■ 0. 
Theorem 6.1. For every a; G §, and h we have that: 

^^ E[h{X-m-h{x) _ h{oo)-hix) 

t\0 t Ix ' 

Proof. We will show that, for any fixed e > 0: 

t\0 t ^x 

li^ nh{X-m-Kx) ^ h{^) - h{x) - e 

t\0 t ~ Ix ' 

from these inequalities, (6.2) follows immediately. We will only show (6.3), since (6.4) is analogous. 

Since ^ G P C Co, we can take S G (0,7^;) and H > such that \h{y) — h{oo)\ < e whenever 
7j, < (5 and sup^^g \h{y)\ < H. 

^M^!M^ = i:(My)-M-))^+ E (%)-M-))^ (6.5) 

y^S.s y&Ss\{x} 
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The second term can be dominated by 2H\F{X='{t) e Ss\ {x}). Since J2yeSs < °° (^Y 
Remark 3.4), we can use Proposition 5.5 to get that this quantity vanishes as t \ 0. 
The first term of (6.5) can be dominated by: 

{h{oo) - h{x) + e)-^ \ J ^ ^ (/i(oo) - h{x) + e) — — 



(/i(oo) - h{x) + e) 



1-Pxx{t) F{X^{t)e§s\{x}) 



t 



We can again use Proposition 5.5 to show that the second term inside the brackets converges 
to zero, while the first term converges to by Proposition 5.6. □ 

Definition 6.1. For x G S, let us denote by "X(t) = x first visit" the event that "at time t, X 
is at its first visit to x"; more precisely, making Lg = and Hf = inf{t > L^_i ■ X(t) = x}, 
Lf = M{t > Hf : X{t) ^x},i> 1, then {X{t) = x first visit} = {X{t) = x, > t}. 
Conversely, "X(t) = x not first visit" is the event {X{t) = x} \ {X{t) = x first visit}. 

Lemma 6.2. If h and sup^^§Xx < oo then: 

lim ^ \h{x) - h{oo)\F {X°°{t) = x not first visit) = 



(6.6) 



Proof. Throughout this proof we will always assume that the process started at oo and omit this 
index of our notations. We will also assume, without loss of generality, that h{oo) = 0. For x G S 
and t>0, let 

N.{t) 

r<->(t)= ^ 

zeS\{x} 1=1 

For arbitrary x G S and t > 0: 



(6.7) 



(r<->(af)<t) 



r<^>(<Tf) 
e * 



> e"M < eEexp 



Since erf is a sum of two independent exponential random variables and F have stationary and 
independents increments, we have that: 



E 



exp< - 



F<->(af) 



E 

-2 



exp < - 



yy^x 



^x'^J + ly 



< 



F<^>(af) 



The event {X°°{t) = x not first visit} is contained in the event {r<=^>(c7f) < t} n {7^rf < t], 
which are independent. Thus, since sup^. A^; < oo: 



-2 



1 



^ \h{x)\F {X°°{t) = X not first visit) < const ^ 



xes 



,2/eS 



t + -fy 



^ 1 _ e-*/T^ 
Y\h{x)\Xx (6.8) 
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The right hand side of this inequahty (except for the constant) can be written as: 

Let us denote by st = YlxeSt ^o^^ t^** st < oo for every t>0 but ^ oo as t \ 0. 
To control (6.9), we will use that (1 — e~*/'''^)/t < l/jx and that \h{x)\/jx is bounded by a 
constant (since h eV). With these considerations we can bound (6.9) by: 

= const ^ = const — 

To control (6.10), we will use that 1 — e~^l^^ < 1 and \h{x)\ < const 7^. Thus we can bound 
(6.10) (except for a constant factor) by: 

If the sum in a; is bounded for every value of t, then this expression vanishes as t \ 0, otherwise 
we can bound this expression as: 



2 

K'yy \ sr^ ^xlx _ Asr^ \ly \ *\o 



> □ 



Elf ET=MS t 

Theorem 6.3. Let us assume that sup^g§ < 00. Then for every HeV we have that: 

Um = Um "'"'-"W. (6.11) 

t\o t x-^00 

Proof. Let us denote by h{x) = h(x) — h{oo) and let L = liniaj^oo -7^- Without loss of generality 
let us suppose that L > 0. 



E[h{X°^{t))] — Moo) ^ 1 ^ h{x)F{X{t) = X first visit) + ^ J] /i(x)P(X(t) = a; not first visit) 

xes xes 

(6.12) 

(recall Definition 6.1). 

Note that the second term of right hand side of (6.12) vanishes as t \ by Lemma 6.2. Since 
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h{x)Xx = 0, if we fix an arbitrary a G S, then we can write the first term as: 

1 T / x™/ X „ . . X 1 T / XX P(X(t) = a first visit) 
- h{x)F{X{t) = X first visit) - Kx)>^x — 

1 hix) 

=- V [XaF{X{t) = X first visit) - A^P(X(t) = a first visit)] 

t — ^ An 

xe§\{a} " 

=] Yl ^ [K^X{t) = X first time, af < a?) - A^P(X(t) = a first visit, cr? < af)] (6.13) 

xeS\{a} " 

+- V ^ [AaP(X(t) = X first visit, af > af) - XxF{X{t) = a first visit, a? > )] . (6.14) 

x&\{a} 

We can bound the absolute value of (6.14) by: 

J |/i(a;)|P(X(t) = a; first visit, (jf >(7f) (6.15) 

x&\{a} 

+~ V |/i(x)|Aa;P(X(t) = afirst visit,(T? >(7f) (6.16) 
A/7 t — ^ 

a;eS\{a} 

Analogously to the proof of Lemma 6.2, we can bound (6.15) by: 

(\ -'^ 
1 ^ Ap^ 

< const Yl f E ' (6-17) 

xes\{a} \yes\{a} / 

where 

rw(0= E E^^^'- (6-18) 

2e§\{a;,a} «=! 

The quantity on the right hand side of (6.17) vanishes as t \ by the dominated convergence 
theorem. 

Given that cr" > erf , then af has the same distribution as a sum of two independent exponential 
random variables, of rates Xx + An and Xa- Using this fact we can bound (6.16) by: 

~ E IM^)|A.P(r<^''^>(c7?)<t,7xrf <i,a?>af) 

" x&\{a} 



< const E \Hx)\Xx '-'"''\ ^\ (— V E ^1 E ^ 



-1 



< const ( E ^] E miK'-^^. 

\yGS\{a} 'y j x&\{a} 

Disregarding constants, the latter quantity is exactly (6.8), which showed up at an intermediate 
step of the proof of Lemma 6.2. Following that proof, we conclude that (6.16) vanishes as t \ 0. 
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Suppose that Sx is a exponential random variable with rate Aa + A^;, independent of the process. 
If we call the distribution function and the density of r^^'"^(S'a;) (as in (6.18)) by Fx and fx 
respectively, then we can write (6.13) as: 



a;eS\{a} 

1 E k 



A. 



Aa + Ax 

Ax 



Jo 



t—s t — s 

e ix — e to- 



ds 



x£S\{a} 



Aa + Ax Jo 

xeS\{a} 

The absolute value of (6.20) can be bounded by: 



Aa + Aa: 



fx{s) 
ffxis) 

Jo 



t-s 

1 — e tx 



1 — e la 



ds 



ds 



(6.19) 
(6.20) 



t - — 

f E \K^)\^J U{s) ^~\ " ds<^ E IM^)|A.F.(t). 



a;eS\{a} 



a;eS\{a} 



This vanishes as t \ by the dominated convergence theorem. 



For an e > fixated, take (5 > such that 



L\ < e whenever 7^ < 5. Integrating (6.19) by 



parts, we obtain: 



_ 1 X - h{x) Xx 

a;eS\{a} 

h{x) Xx 



Fx{s)e tx ds 



= -- T [ Fx{s) 



t-s 

e ix ds 



h{x) Xx 



* r-tX't 1 + Jo 

xeSs\{o-} 



Fx{s)e -Ix ds 



(6.21) 
(6.22) 



Note that \h{x)\/'yx is bounded since h eV. Thus we bound the absolute value of (6.22) by: 

const E -^^7 / ^x{s)ds. 
xeSs\{a} •'^ 

This quantity vanishes as t \ since V^^'°'^ {Sx) is a continuous positive random variable and 

Y.X&S < 00. 

We can bound (6.21) above and below by —{Li: e)G{t)/t, where 



G{t):= E 



^0S5\{a} 



Xxlx 
Aa + A3 



r 1 * 

/ Fx{s)-e-' 

Jo Ix 



tx ds. 



Now we are left with showing that G{t)/t ^ 1 as t \ 0. 

The integral in the definition of G is a probability distribution function, since it is a convolution 
of a probability distribution function and a probability density function. Thus G is a distribution 
function of a finite measure. Our strategy now is to use Karamata's Tauberian Theorem (as stated 
in Theorem XIII. 5.1 of Feller (1971)), that relates the behavior of the G near the origin with the 
behavior of its Laplace transform near +00. 
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Let us denote the Laplace transform of G by G. For y3 > 0, integrating by parts, we can write: 

/"OO f'OQ 

G{P) := / e-^^G{dt) = P e-^^G{t)dt 
Jo Jo 

\ poo pt ^_ 

= V -^^P / e-*^ / F,{s)e-^dsdt. (6.23) 

Note that the integral in t at the right hand side of (6.23) is the Laplace transform of the 
convolution of two functions. We can compute the Laplace transform of each one of them, getting: 



7x 



poo 

Jo 



e-/'*F.(t)di=iE[e-/^r<-''>M=^4^(Ai + A. + /3 ^ 

yeS\{a,x} 



Thus 



x^Ss\{a} 1^ 1^ \ y&S\{a,x} ^ 

Since sup^g§ < oo and J2xeSs ^ ^® — > 1 as /3 — )■ oo. We have thus 
verified the assumption of the Tauberian Theorem: 

GjaP) p^oo I 
G{/3) 'a 

yielding that G{t)/G{l/t) ^ 1 as i \ 0. With this we can conclude: 

G(t) _ G(t)/G(l/t) iM)^ ^ ^ 

' " (i<i(i))"' 

Corollary 6.4. Theorem 6.3 also holds when svop^^^Xx = oo. 

Proof. (Sketchy) The basic idea is to break sites with large Aa; into several sites with bounded A^;. To 
formalize this argument, we will construct simultaneously a K process with the original parameters, 
and another K process related to the first, but with bounded weights. 

Take S' = {{x, n) G S x N : n < Xx}; this is a countable set. For {x, n) G S' let us define: 

\i Ax I 

It can be verified that {A^^. 7^^. : {x,n) G S'} satisfy (1.1). Furthermore: 

Kx,n) = ^x, sup A'(^ ) < 1 

Since the superposition of independent Poisson processes is a Poisson process, we may construct 
{X{t), t >0} and {X'{t),t > 0}, K processes started at oo in the same probability space such that: 
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• X has S as the state space and parameters {X^, '■ x ^ S}; 

• X' has S' = S' U {00} as the state space and parameters {A'^^ n)' ^(i n) ■ ^) ^ 

• -^(t) is the projection in the first coordinate of X'{t). 

Let T> and V be the domain of the generator as defined in (6.1) for the processes X and X' 
respectively. 

Take h e V; we want to show (6.11). For that let us define /i' : S' ^ M by h'{{x,n)) = h{x), 
h'{oo) = h{oo). With this construction we have that h{X{t)) = h'{X'{t)) a.s. . 

It may be readily verified that h' eV, so we conclude using Theorem 6.3 for the process X': 

^.^ E[hiX{t))]-hioo) _ ^.^ E [h'{X'{t))] - /i'(oo) 



t\0 t t\0 t 

^.^ h'{oo)-h'{{x,n)) ^ ^.^ hjoo) - h{x) ^ 

Theorem 6.5. The operator A:T> ^ Cq. Given by: 

(f(oc)-m ifxeS; 

^f(^^ = \y " /oo)-/(.) ' (6.24) 
[ lima;_j.oo ' , tfx = oo, 

has a closure that is the Markov generator of the semigroup of the K process over Cq. 

Proof. Following Theorem 2.6 from the Chapter 1 of Ethier and Kurtz (1986), equipping Cq with 
the supremum norm, making it a Banach space, we need to verify that: 

1. P is dense in Cq; 

2. 11^/ - AfW > m\\ for every ^3 > and / G P. 

3. The range of / — .4 is Cq, where I is the identity operator. 
Let us prove each item separately: 

1. Given a G Co and a e > 0, we want to find a f eV such that sup^-gg \ f{x) — g{x)\ < e. Since 
g G Co, there is a (5 > such that \g{x) — 5(00)! < e/2 whenever jx < S. 

Fixing such S, let us define /(oo) = 5(00) and f{x) = g{x) whenever 7^, > S. By (1.1) we 
have that ^x-'yx>S ^ Ylx-'yx<S = 00, so we can choose finitely many x not yet 
chosen, and define f{x) in a such a way that Xx{f{x) — /(oo)) = 0, if we set f{x) = g{oo) 
for every other x. One may readily check that such an / satisfies our conditions. 

2. Take / G P and /3 > 0. 

Since J2x&ifi^) ~ f{'^))^x = 0, then ||/|| > |/(oo)|, so for a fixed e > we can take y G S 
such that \ fiy)\ > ||/|| - e and |/(y)| > |/(oo)|. This implies that /(y) and -Af{y) have the 
same sign, so we may conclude that 

wf - AfW > \my) - Afiy)\ > \my)\ > m\\ - p^- 
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3. Given a 5 G Co, let us define: 



L := 



„ 1 + 



.xes 



+ 7: 



-1 



^x'Jx 



fix) 



.xen 

5(00) + L, otherwise. 
Direct computations verify that / G P and / — Af = g. 



+ 7= 



i9ix)-gioo)) 



□ 
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